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ON THE FALSE DISCOVERY RATE AND AN ASYMPTOTICALLY 
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In this paper we introduce and investigate a new rejection curve 
for asymptotic control of the false discovery rate (FDR) in multi- 
ple hypotheses testing problems. We first give a heuristic motivation 
for this new curve and propose some procedures related to it. Then 
we introduce a set of possible assumptions and give a unifying short 
proof of FDR control for procedures based on Simes' critical values, 
whereby certain types of dependency are allowed. This methodology 
of proof is then applied to other fixed rejection curves including the 
proposed new curve. Among others, we investigate the problem of 
finding least favorable parameter configurations such that the FDR 
becomes largest. We then derive a series of results concerning asymp- 
totic FDR control for procedures based on the new curve and discuss 
several example procedures in more detail. A main result will be an 
asymptotic optimality statement for various procedures based on the 
new curve in the class of fixed rejection curves. Finally, we briefly 
discuss strict FDR control for a finite number of hypotheses. 

1. Introduction. The false discovery rate (FDR) has become one of the 
main research objects in multiple decision theory during the last decade. One 
reason for the increasing interest in FDR controlling procedures is the need 
for a suitable error rate controlling criterion if the number of hypotheses is 
very large. It is widely accepted that strong control of the familywise error 
rate (FWER) is far too restrictive for large systems of hypotheses. Beginning 
with the pioneering paper [2] on a linear step-up FDR controlling procedure 
based on Simes' test (cf. [22]) for independent p- values, meanwhile a series 
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of alternative procedures not only for independent but also for dependent 
test statistics have been introduced and theoretically investigated. For the 
latest developments we refer to [1, 5, 6, 7, 8, 16, 18, 20]. 

In general, the original linear step-up (LSU) procedure does not exhaust 
the prespecified FDR level, which gives hope for improvement with respect 
to power. In this paper we are concerned with the problem of finding an in 
some sense optimal rejection curve when the number of hypotheses tends to 
infinity. 

The paper is organized as follows: In Section 2 we introduce some notation 
around the FDR and a basic model which will be considered throughout the 
paper. Moreover, we briefly describe the interaction of rejection curves, crit- 
ical value curves and the empirical cumulative distribution function (ecdf) 
of p- values. In Section 3 we give a heuristic derivation based on asymptotics 
of the new rejection curve denoted by fa and introduce some possible pro- 
cedures based on this curve. Section 4 is of more general character and deals 
with assumptions, methods of proof, least favorable configurations (LFCs) 
for FDR control and upper FDR bounds. We first introduce a series of pos- 
sible assumptions and give a unifying short proof of FDR control for proce- 
dures based on Simes' critical values which, among others, summarizes the 
derivations in [4, 19] for dependent p-values (or test statistics) in an elegant 
way. Then we investigate the problem of finding LFCs for the FDR, that 
is, we look for distributional settings where the FDR becomes largest and 
derive upper bounds for the FDR. Section 5 is concerned with asymptotic 
FDR control for procedures based on under the assumption of indepen- 
dent and uniformly distributed p-values on the unit interval with respect 
to the set of true null hypotheses. Moreover, we formalize the asymptotic 
optimality of fa- Some concluding remarks, including a brief discussion on 
some properties of procedures related to fa if a fixed number of hypotheses 
is at hand, are given in Section 6. 

2. Notation and preliminaries. Throughout the paper, we use the follow- 
ing notation: Let {X, A, {P^ : 1) G ©}) denote a statistical experiment and let 
{Hn)neN be a sequence of null hypotheses with 7^ Hn C Q. The correspond- 
ing alternatives are given by = 6 \ Hn- Let (pn)neN denote a sequence 
of p- values with pn : (^,-4) — > ([0, l],B), where B denotes the Borel-a-field 
over [0, 1]. Denote the set of positive integers by N and let = {1, . . . ,n}, 
Io = Io{^) = {i£n--^(£Hi}, Ii =Ii(^)=N\/o = {iGN:i9^i?i} and I„j = 
^n,j{^) = Ij n N„, j = 0, 1. As usual, let a p- value pi for testing Hi satisfy 
Q < P^{pi < x) < X for all ?9 G -ffj, i G N and x G (0, 1]. We assume that the 
measurable space {X-,A) is large enough to accommodate probability mea- 
sures P/o, /q C N, under which all p- values pi,i G /q, are i.i.d. uniformly 
distributed on [0, 1], and all pi, i Ii, follow a Dirac distribution with point 
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mass 1 in 0. We refer to Pig as a Dirac-uniform configuration. We also as- 
sume that for every G and i € loii}) there is a probability measure P^i 
defined on {X,A) for which the sequence (pn)nGN has the same distribution 
under P^i as the sequence {Pn)neN under P^, the only difference between 
these two sequences of p-values being that = 0. This is a technical as- 
sumption which will be used in Section 5 for the determination of upper 
bounds for the FDR. Notice that the Pj^ 's and P^i 's need not be contained 
in {P^:i?€0}. Under each Pj^, the Extended Glivenko-CantelU theorem 
(cf. [21], page 105) applies for the ecdf F„ (say) of the values, that is, 



(2.1) lim sup 

"-*°°tg[o,i] 



F„(t)-f^ + ^t 
\ n n 



[P, 



where Uj = nj{n) = \In,j\-, j = 0, 1. For a nonempty set Iq ^ N we denote 
by /q the set /q \ {min/o} in the sequel. 

For notational convenience we introduce Fn,j^ i = 0, 1, as the ecdfs of the 
p-values corresponding to the true (j = 0) and the false (j = 1) hypotheses, 
respectively. Now let Tin = {Hi : i G N„} and let = {ipi : i € N„) denote 
a non-randomized multiple test procedure for Tin- For a fixed G and a 
given test we define the number of false rejections hy Vn = \ {i & f^n ■ '^i = 
1 and G Hi}\ and the number of all rejections by Rn = \{i G N„ : = 1}|. 
Then the actual (expected) FDR of a multiple test ip^^^) given G G is 
defined by 



FDR^((/j(„)) = E^ 



Rr,yi 



The ratio Vn/[Rn V 1] is the false discovery proportion (FDP). A multiple 
test (/9(„) is said to control the FDR at level a G (0, 1), if 

supFDR^((/P(.„)) < a. 

The original LSU procedure for independent p- values if^^n)^ for short) rejects 

Hi iff Pi < ma/n, where m = max{i G Nn :pi:n < «i^n^} with a}!^ = ia/n 
for i = 1, . . . ,n and pi:n < • • • < Pn.n denoting the ordered p-values. This 
procedure can be rewritten in terms of the ecdf Fn of the p-values. Let 
= sup{t e [0, 1] : Fn{t) > t/a}. Then (/jgu ^^jg^^g ^. p. < t((^LSU)_ 

The rejection curve ra{t) =t/a will be called the Simes line. Notice that 
'^f-ri^ ~ ''^a^i'^/^)- Now let 1? G and suppose that the pj's, i G In,o{^), are 
i.i.d. uniformly distributed on [0, 1]. Then one of the most interesting results 
for the LSU procedure is that 



(2.2) 
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Different proofs of this fact can be found in [4, 11, 19, 23]. In [2], it was 
only shown that the FDR is always bounded by riQa/n for the LSU pro- 
cedure. However, the fact that the FDR is bounded by riQa/n raised hope 
that improvements of the LSU procedure should be possible. In [13], an im- 
provement taking a Bayes risk approach has been developed. Another idea 
is to estimate the number no of true null hypotheses and to adjust the LSU 
procedure in order to exhaust the prespecified FDR level. This approach 
has been worked out in [23]. Multi-stage adaptive methods were recently 
proposed in [3]. The introduction in [14] gives a good overview about the 
development of several (adaptive) approaches aiming at improving the LSU 
procedure. In this paper we tackle the problem in a more direct way by de- 
riving a new rejection curve which will serve as a basis for various stepwise 
multiple test procedures. 

Many multiple test procedures can be described in terms of the ecdf F„ 
of the p-values and a rejection curve r. Let p : [0, 1] — [0, 1] be nondecreasing 
and continuous with p(0) = and positive values on (0, 1] . Define critical 
values ai;n = p{i/n) G (0, 1] for z = 1, . . . , n. We call p a critical value function. 
Moreover, r defined by r{x) = infju G [0, 1] : p{u) = for x € [0, 1] (inf = 
oo), will be called a rejection curve. For illustrative purposes, a plot of Fn 
together with the rejection curve r is most useful in order to demonstrate 
the decision procedure. Note that we have the following relationship between 
the ecdf of n distinct p- values pi, . . . the ordered p- values, the critical 
values ai:n = pii/n) and the rejection curve r: 

Fn{Pi:n) > r{pi:n) if and only if Pi:n < Oi-n- 

A point t = pi.,n satisfying Fn{pi:n) > r{Pi;n) and Fn{pi+i:n) < r(pj+i;„) is 
called a crossing point (CP) between Fn and r. In this paper, we consider 
multiple test procedures where one of the CPs is chosen as a threshold t* in 
order to reject all Hi with pi <t*. Other thresholding rules are extensively 
studied in [1]. It is immediately clear that the proportion (i?„ — Vn)/{ni V 1) 
of rejected false null hypotheses with respect to all false null hypotheses is 
nondecreasing in the threshold t* . Therefore, we look for procedures which 
maximize t* for any given set of p-values subject to FDR control. Often, 
E^[(i?„ — Vn)/{ni V 1)] is defined as power. Loosely speaking, larger thresh- 
olds lead to larger power. 

3. Heuristic derivation of a new rejection curve. In order to derive a 
new rejection curve we consider Dirac-uniform configurations Iq C N. 
This policy is motivated by the fact that Dirac-uniform configurations often 
provide upper bounds for the FDR, see Section 4. Let = nQ{n)/n denote 
the proportion of true hypotheses among the first n hypotheses. We refer 
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to this situation as the Dirac-uniform finite model, DU„(Cr!,)- Now suppose 
that 

lim c„ = Ce[o,i]. 

n^oo 

Then (2.1) imphes that, for n tending to infinity, the ecdf Fn of the observed 
p-values converges to 

Foo(t|C) = (l-C) + a foraUte[0,l] [Pi,]. 

This situation wih be cahed the Dirac-uniform asymptotic model, DUoo(C) 
for short. Now suppose we choose some t £ (0, 1] and reject all hypotheses 
Hi with Pi < t. Then, in the DUoo(C) model, an expected proportion of tC, of 
the true hypotheses will be rejected, that is, the false discovery rate FDR^(t) 
(say) is given by 

FDRf(t) 



(i-O + tC 



Now we determine a G [0,1] such that FDR^(t^) is exactly a for some 
prespecified a G (0, 1). Obviously, this can only work for Q G [a, 1). In this 
case we obtain 



«(i-C) 

C(l-a)' 



For C, G [0,a) one may set = 1, which implies that all hypotheses are 
rejected and FDR^(l) = C, <a. 

We now look for a strictly increasing rejection curve fa '■ [0, 1] — > [0, 1] 
which does not depend on C, but tells us which of the hypotheses can be 
rejected such that the limiting FDR is exactly a for C, G [a, 1). The function 
/q, can be found by requiring fa.{tc) = Foo{t(^) for all Q G [a,l), which is 
equivalent to 



fa 



a(l-C)\ 1-C 



C(l — a) J I — a 



Obviously, 



i(l — a) + a 



fulfills this requirement. In the DUoo(C) model, the limiting ecdf -Foo('lC) 
crosses fa in the point (t^, y^^) with t( = min{Q(l — () / [({1 — a)], 1} and = 
min{(l — C)/(l ~ 0^)5 1} foi' the first time, and it should be noted that this 
is also true for ^ = 1 if we define (ti, yi) = (0, 0). 



6 



H. FINNER, T. DICKHAUS AND M. ROTERS 



Remark 3.1. Comparing the Simes hne ra{t) =t/a and the new rejec- 
tion curve fait), we obviously have ra{t) > fa(t) for t > and the derivative 
in i = is 1/a for both curves. Moreover, notice that fa obeys the symmetry 
property 

fa\t) = ^ ,f . = 1 - fail - t) for all t G [0, 1], 
1 — (1 — a)t 

where f~^ denotes the inverse of fa- Clearly, f~^ is a critical value function. 

The new rejection curve fa will be called the asymptotically optimal 
rejection curve (AORC). The question is how to implement fa not only in the 
Dirac-uniform models but also in more general models into a multiple test 
procedure which controls the FDR level a strictly or at least asymptotically. 
The critical values induced by fa are given by 

1 f i\ ii/n)a ia 

Remember that distinct p-values satisfy 

FniPim) > faiPi-.n) if and Only if Pi:„ < ai:„. 

It is tempting to use cxi-n ^ ' ■ ■ Oifi-n in a step-up (SU) procedure for test- 
ing n hypotheses. Unfortunately, On-.n = 1, so that this procedure always 
rejects all hypotheses. This pitfall is due to the fact that /a(l) = i^n(l) = 1- 
Therefore, we need some adjustment with respect to fa or the SU procedure. 
In the remainder of this section, we consider some candidates for asymptotic 
FDR control avoiding the aforementioned pitfall. 

Example 3.1 (Step-up-down procedures). An interesting class of pro- 
cedures based on critical values < ai-n < • • • < an:n < 1 are step-up-down 
(SUD) procedures introduced in [24] and studied in [19] in terms of FDR 
control. For A„ G N„ an SUD procedure ^f^^ = i^i, • • • , ^n) of order A„ is 
defined as follows: If pA„:n < aA„:n, set m„ = max{j e {Xn, ■ ■ ■ ,n} -.pi-.n < 
ai-n for all i G {A„, . . . , j}}, whereas for px^-n > ax^-n, Put m„ = sup{j G 
{!,..., Xn}:pj:n < aj:n} (sup0 = -oo). Define Lfi = I if Pi < am^-.n and 
(pi = otherwise (a_oo:n = — oo). Note that = 1 yields a step-down (SD) 
procedure and Xn = n yields an SU procedure. The order of an SUD pro- 
cedure can be defined in terms of a fixed parameter A G [0, 1] by setting 
A„ = inf{j G Nn : aj:n > A} (inf = n). Then A = (A = 1) corresponds to 
an SD (SU) procedure. An SUD procedure of order A„ = A„(A),A G [0,1), 
based on fa resolves the problems around the point t = 1 in an elegant way. 
It is obvious in view of Remark 3.1 that in the case of A > a the new SUD 
procedure based on fa rejects at least all hypotheses rejected by the LSU 
procedure, possibly more. Therefore, one cannot expect that the FDR level 
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is controlled in the finite case. However, it will be shown that the FDR is 
controlled asymptotically. Note that is component- wise nondecreasing 

in A. 



Example 3.2 (Adjusted SU procedures based on As noted before, 

an SU procedure based on ai-n = fa^{i/n) cannot work. Therefore, some 
adjustment of fa in an SU procedure is necessary. We first consider the 
case where the adjusted rejection curve satisfies that f^^{x)/x is non- 
increasing in X, an important property for the calculation of the FDR. One 
may specify some k E (0, 1) and define a new rejection curve 



fa{x), 0<X<K, 
h{x), K<X<1, 



such that fa'^^{x)/x is nonincreasing in x, fa'^^{x*) = 1 for some x* < 1, and, 
fain) = For example, one may choose h{x) = ax + b for suitable values 
a, 6 > 0. We consider two possible choices of h {hi and /12 say) and refer to 

(i) 

the resulting rejection curves as /A7^,i = l,2. Let 

hl{x) = fa{K){x - k) + fa{K), X£[k,1]. 

Then h[iK) = /^(ac), = /^(k) and hi{x*) = 1 for x* = k{1 - a)(2 - 

k) + a < 1. The largest possible slope of /i is a = /^(k)/^. This leads to the 
second choice, that is, h2(x) = xfain)/ k. This is close to the truncated SU 
procedure in Example 3.3 below. Note that h2{x*) = 1 for x* = k(1 — a) + a. 
For example, suppose that k = fa^{i/n) for some fixed i G N„. Then the 
step-up critical values are given by 



lr.n=\ J ^ 

nfa{K)' 



faHj/n), l<j<i, 

i + 1 < j <n. 



Example 3.3 (Truncated SU procedures based on Let k G (0, 1) be 

fixed and define 



Paix) 



f~Hx), 0<x</„(k), 

K, fain) <X<1. 



With ji-n = ™iii{/Qr^(V^); we have jj-^n = Pa{i/n) iov j = i, . . . , n. Hence, 
the truncated SU procedure is well defined in terms of pa- It is worth men- 
tioning that this type of procedure differs substantially from the adjusted 
procedures discussed in Examples 3.1 and 3.2, because the monotonicity 
behavior of the ratio pa{x)/x changes at x = fai^), which makes FDR cal- 
culation much subtler. 
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The next section deals with methodologies for proving FDR control and 
deriving upper FDR bounds. The results will be applied in Section 5 in order 
to derive conditions for asymptotic FDR control for procedures based on the 
AORC. The aforementioned example procedures are then investigated at the 
end of Section 5 with respect to asymptotic FDR control and asymptotic 
optimality. 

4. A unifying proof of FDR control and upper FDR bounds. Suppose 
that Rn and respectively, are defined in terms of p- values pi,...,pn 

and critical values Oi-n = pii/n) for some critical value function p. Consider 
the following three sets of possible assumptions. The first two assumptions 
concern the structure of the test procedure (test assumptions): 

(Tl) Vi € N„, -.pi < ai:n implies ipi = 1. 

(T2) Vj € N„ : i?„ = j imphes Vf G N„ : [(^j = 1 <^ pi < aj:n] ■ 

The second set of assumptions concerns properties of distributions of p- 
values and Rn (distributional assumptions): 

(Dl) V?? G : Vj G N„ : Vi G /„,,o('!9) : P^iRn > j\Pi < t) is nonincreasing in t G 
(0,aj:„]. 

(D2) W G e-.yj G N„:Vi G /„,o(^?):Vt G (0, a,,„] : P^(i?„ > j\pi < t) < 

PARn>j). 
(D3) Vt? G e : Vi G Infl{^)■■P^ ~ C^([0, 1]). 

Finally we have two possible independence assumptions: 

(11) V??Ge: The Pi's, iG/n,o(^9), are i.i.d. 

(12) V?? G : (pt : i G /ri,o) and {pi-i & In,i) are stochastically independent 
random vectors. 

Furthermore, we often refer to step-up tests or step-up-down tests of order 
A^. The simple structure of SU tests often simplifies derivations concerning 
properties of these tests. If (/9(„,) is a step-up-down test, the properties of 
a step-up test remain valid in the step-up branch of such a procedure. For 
example, it is important to note (see [19], page 248) that given a step-up- 
down test of order A^, under (D3), (II) and (12), we get for all G and 

(4.1) Vj = 1, . . . , A„ : yt G (0, aj.,n] : P^{Rn > j\Pi <t)= P^Rn > j), 

(4.2) Vj = 1, . . . , A„ : Vt G (0, ay.n] : P^(i?„ = j\pi <t)= P^. (i?„ = j). 
For An = n, that is, for a step-up test, we even get 

Vj = 1, . . . , n : Vt G (0, Uj-.n] : P^{Rn > j\Pi <t)= P^. (i?„ > j). 
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Assumptions (Tl) and (T2) concern more general structures of test pro- 
cedures. Step-up-down tests satisfy both (Tl) and (T2). Tlie monotonic- 
ity assumption in (Dl) is somewhat weaker than the PRDS assumption 
(PRDS: positive regression dependency on subsets). More precisely, from 
the -almost sure antitonicity of the factorized conditional probability 
P-d{Rn > j\Pi = t) in t G [0,aj:„] we obtain the property formulated in (Dl), 
where the equality in the condition is replaced by an inequality. This type 
of conclusion is indicated in [17] and can be proved in an easy way by using 
Wijsman's inequality, see [25]. Anyhow, (Dl) is the decisive condition for 
dependent values in order to prove FDR control or to derive upper bounds 
for the FDR. Examples of distributions being PRDS are extensively studied 
in [4] and [19]. Anyhow, we have no example with dependent p- values yet 
where (Dl) is easier to show than PRDS. However, under the independence 
assumptions (II) and (12), the important class of SUD tests satisfy (Dl). 
Property (D2) will only be used under the independence assumptions (II) 
and (12) and is an important tool for deriving LFC results. In case of de- 
pendency, (D2) is often violated. Assumptions (D3) and (II) concern the 
distribution of p- values under the corresponding null hypotheses. 

The following theorem and its proof unify, simplify and slightly extend 
the results and the proofs given in [4] and [19], respectively. 

Theorem 4.1. Let a S (0,1) and let he a multiple test procedure 
for Tin defined in terms of Simes' critical values Oi-n = ia/n, i = 1, . . . ,n. 
Let E such that uq hypotheses are true and the remaining ones are false. 
Lf (Tl), (T2) and (Dl) are satisfied, then 

riQ 

FDR^(V9(„)) < —a, 

with "=" if f(^n) step-up test and (D3), (11) and (12) are satisfied in 
addition. 

Proof. Consider the following chain of (in)equalities: 
FDR^(<^(„))= E-^^(i?n=J>* = l) 

n ^ 

(4.3) = ^ Y-P&{Pi<aj:n)Pi}iRn=j\Pi<aj:n) 

ie/„,o(i?) i=i 

n 

(4.4) < Yl T.^P^i^n=J\p^<a,..n) 
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(4.5) 




i 7-1 



P-d{Rn>j\Pi < aj-n) 



(4.6) 



= — a. 

n 



Equation (4.3) holds under (T2), and "=" holds in (4.4) if (D3) holds. 
Inequality (4.5) holds under the assumption (Dl) with "=" if ipf^^-^ is a step- 
up test and (D3), (II) and (12) hold. Finally, (4.6) is a consequence of (Tl). 
□ 

Remark 4.1. The key step in the proof is (4.5), where P^{Rn > j\Pi < 
aj-i:n) is replaced by P^{Rn > j\pi < aj-^n) for j = 2, . . . ,n according to 
assumption (Dl). In case of dependency or in case of a non-step- up test, the 
difference between these quantities may sum up to a considerable amount, 
that is, the FDR may be much smaller than the upper bound noa/n in such 
cases. For a detailed investigation of the latter phenomenon, see [9]. 

One of the main problems in order to ensure FDR control of a multiple test 
procedure is to find least favorable parameter configurations (LFCs), that is, 
parameter configurations under which the FDR for a given test procedure 
becomes largest. Obviously, LFCs are no issue for the LSU procedure if 
(D3), (II) and (12) hold true. To date it looks like SU procedures are easier 
to cope with than SD or SUD procedures. One reason for this is that Dirac- 
uniform configurations can often be viewed as least favorable for certain SU 
procedures. This fact is based on the following important result: 

Theorem 4.2 (Benjamini and Yekutieh (2001); cf. [4]). An SU proce- 
dure with critical values ai-n < • • • < dn-.n fulfilling (D3), (11 ) and (12) has 
the following properties: 

(1) If the ratio ai;n/i is increasing in i, as {pi : i E In,i) increases stochas- 
tically, the FDR decreases. 

(2) If the ratio oti-n/i is decreasing in i, as {pi'.i In,i) increases stochas- 
tically, the FDR increases. 

Hence, under the assumptions of Theorem 4.2, the Dirac-uniform config- 
urations, where all p- values under alternatives are almost surely 0, can be 
viewed as LFCs if the ratio cui-n/i is increasing in i. More precisely, on the 
parameter subspace, where exactly no (ni) hypotheses are true (false), the 
FDR becomes largest if the p-values under alternatives are almost surely 0. 
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Therefore, it suffices to consider all Dirac-uniform configurations in order to 
check whether the FDR is controlled at level a. 

Unfortunately, the method of proof given in [4] does not seem to work for 
SD and SUD procedures. However, we show below that Dirac-uniform con- 
figurations often provide upper bounds. To this end, we define q{x) = p{x) jx 
for all X G (0, 1] and assume that gr(0) = limsup2..|o < Moreover, we 
define the (continuous) function g by ^(x) = maxo<t<x q{t), X G [0, 1]. Hence, 
q is the upper isotonic envelope or, in other words, the least isotonic majo- 
rant of q. For the derivation of upper FDR bounds, we now introduce the 
following additional conditions: 

(Al) If {pi, . . . ,pn) is stochastically not greater under t?i e than under 
£ 0, then is stochastically not greater under t?2 € than under 



(A2) p{x)/x is nondecreasing for x G (0, 1]. 

Note that oti-n/i is nondecreasing in i if (A2) holds. If p is differentiable 
on (0,1), (A2) is equivalent to p'{x) > q{x) for x G (0,1). In what follows, 
q is essential in deriving upper bounds for the FDR. Note that q^q for 
the truncated step-up procedure introduced in Example 3.3. If qi^q, the 
bounds for the FDR based on q may not be that sharp. 

Theorem 4.3. Let G such that hq G N^, hypotheses are true and 
the remaining ones are false. Let iq = min/o [and /q = /q \ {io} as defined 
below equation (2.1)]. If (Tl)-(12) are satisfied, then 



Proof. Let bj = P^{Rn > j\pio < aj-.n) and Aq{j/n) = q{j/n) - 
q{[j — 1) /n) for j = 1, . . . , n. Then, proceeding as in the proof of Theorem 4.1 
we get for fixed ?? G under (D1)-(D3), (II) and (12) 



i9i G0. 



(4.7) 



FDR^ ((/?(„)) <—^q{j/n)P^^Q{Iin/n=j/n) 




n 



FDR^,((^(„)) 



n 



■n 



j\Pio <aj:n) 



n 



(4.10) 




J2q{j/n)P^{R, 



■n 



j\pio <aj:n) 
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(4.11) < ^ 

n 



(4.12) < ^ 

n 



q{l/n)bi+J2^QU/n)bj 

n 

qil/n)P.^., (i?„ > 1) + ^ Aq{j/n)P^., {R^ > j] 



no 
n 



<iU/^)P^H^ {Rn/n = 2 In), 

i=i 

which proves (4.7). In view of P^i^ {Rn > 0) = 1 according to (Tl), (4.8) fol- 
lows immediately. If is a step- up test, which implies (4.1) for A„ = n, 
we have equality in (4.11) and (4.12), hence q = q yields equality in (4.10). 
Finally, in order to prove (4.9), we use the same argumentation as in the 
proof of Theorem 4.2 given in [4], that is, that stochastic increase in the 
distribution of the random vector (pi,...,p„) can be characterized by the 
increase of the expectation of all nondecreasing functions (in case the expec- 
tation exists). To this end, we note that obviously Rn = |{« € 'Nn'-^i = 1}| 
is a nondecreasing function of and hence, due to (Al), is stochastically 
nonincreasing in (pi, . . . ,Pn)- The isotonicity of q completes the proof. □ 

Inequality (4.9) will be a helpful tool in order to calculate upper FDR 
bounds and to prove FDR control, because it only makes use of the distri- 
bution of Rn under Dirac-uniform configurations. Especially for SUD tests, 
this distribution can be handled analytically. 

5. Asymptotic FDR control for procedures based on the AORC. This 
section deals with conditions for asymptotic FDR control for procedures 
based on the new rejection curve. A major result will be that the example 
procedures presented in Section 3 control the FDR asymptotically. Theo- 
rems 5.1 and 5.3 provide sufficient conditions for asymptotic FDR control. 
If the underlying procedure leads to a determinable proportion of rejected 
hypotheses. Theorems 5.2 and 5.4 even give explicit values for the resulting 
FDR. Moreover, the asymptotic optimality of fa is formalized in Theo- 
rem 5.5. 

Theorem 5.1. Suppose is based on p < and that (^TlJ-('I2j 
and ( A.1) are fulfilled. If for all nonempty sets /q C N and all subsequences 
{nk)km Q N with linifc^ooCnfc = C for some ( £ [0, 1], it holds that 

(5.1) limsup^</,(tc) [Pji^], 

fc— »oo nk 

then 

(5.2) limsupsupFDR^(93(„)) < a. 

n.->oo ^ge 
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Vn G N:supFDRi)(vJ(„)) < max — E„ i 

,9cP) l<no<n n 



Proof. Let, for notational convenience, Pm,n refer to a Dirac- uniform 
configuration such that the first m p-values are i.i.d. uniformly distributed 
and the remaining ones follow a Dirac distribution with point mass in 0, 
<m <n, ?iGN. Then we have from inequality (4.9) 

^gQ " ~ l<no<n n 

Since for each n E N the maximum in this inequality is attained at some 
value no(n) (say), we get 

limsupsupFDR^(99(„)) < limsupCn^no{n)-i,nQ{Rn/n), 

n— ♦oo i9g0 n^oo 

where Cn = 'nQ{n)/n, n € N. We now may extract a subsequence {nk)km of 
N with lim/fc_>oo Cn^ = C for some C, G [0, 1] such that 

limSUpC„IE„o(„)_i,„g(i?„/n) = lim CnfeIEno{nfe)-l,nfc^(^"*/'^fc) 
n^oo fe^oo 

< ChmsupE„Q(„^)_i^„^g*(i?„Jnfc), 

fc— ►oo 

where q* denotes the g-function corresponding to the critical value function 
f~^. Similarly as in [11], pages 1003-1004, we are able to select from {nk)k£N 
a further subsequence (without loss of generality with the same name) and 
construct a global set / C N with the property \I D In^ \ = no(^^fc) for all 
A; € N. (At this point it should be noted that the definition of the sets 
at the bottom of page 1003 in [11] has a typo at its right end in that the 
term k{nk) has to be replaced by n/j.) Now we obtain from (5.1) 

ClimsupE„Q(„^)_^„^g*(ii„^/nfe) = ClimsupE//g*(i?„j^/nfe) 

fc— ♦oo fc— ♦oo 

< C&rq* { hm sup Rn^ / Uk 

\ fc— ♦oo 

= cr(/a(tc)) 

= minjo!, (^} < a, 
hence the assertion of the theorem, that is, (5.2) follows. □ 

If we sharpen assumption (5.1), we can even give explicit values for the 
FDR. 

Theorem 5.2. Let G 6, be based on p< and assume (T2), 
(B3), (11) and 

(5.3) lim Cn = Ce[0,l]. 

n— ♦oo 

Iflimn^ooRn/n = r*[P^] for some r* G (0, /^(t^)], then it holds 

(5.4) lim FDR4(^(„)) = Cp{r*)/r* = Cq{r*) < min{a, C}. 
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Proof. From (T2) and for nQ,n € N we get the representation 

Vn = noFn,o(p(^n/n))l{R„>o}- 

From this we obtain the inequahty chain 

\Vn/n- CnP{Rn/n)\<Cn\Fnfl{p{Rn/n)) - p{Rn/n)\< sup |F„.o(t)-t|. 

ie[o,i] 

Hence, using the Ghvenko-Cantelh property (2.1) together with the re- 
maining assumptions of the theorem and the continuity of p, we finahy 
see that Vn/n converges P^-almost surely to C,p{r*). Thus, due to r* > 0, we 
have Wm.n^ao'^'dWn / {Rn V 1)] = Cp{r*)/r*. The right-hand side inequahty 
in (5.4) is obtained by noting that Cfa^i't)/^ is increasing in t E {0,fa{t(^)] 
to Ct(;/fait(;) = mm{a,C} at t = fa{ti;). □ 

The remaining case r* = is treated in the following two theorems. 

Theorem 5.3. Let t9 € G, be based on p< f^^ and assume (Tl)- 
(12), (Al), (5.3) and 

(5.5) Ve>0:liminf inf (t-F„(p(t))) >0 [P^^l. 

n^oo e<t<l 

Then it holds that 

(5.6) limsupFDR^(v3(„)) < Climsupg(x) = Cg(0) = Cg(0) < C"- 



Proof. To avoid triviality, we assume Io{'&) 7^ 0- Then, from (4.7) 
and (4.8) we have that 

(5.7) lim sup FDR^ ((/?(„)) < ClimsupE^io^(-^riM)- 

Since due to (Tl) and (T2) we have for all n € N 

Fn{p{Rn/n)) =Rn/n, 

(5.5) implies that for every fixed e > we obtain limsup„_»oo i?„/n < e P^- 
almost surely, that is, lim^^oo -Rn/'^ = P^-almost surely. Now, since for 
all n S N the maximum absolute difference on the unit interval of the ecdf 
Fn [corresponding to the sequence of p- values {pn)neN] and the ecdf F^° 
[corresponding to the sequence of p- values {Pn)neN defined in Section 2] 
is at most 1/n, condition (5.5) also holds P^io -almost surely, which entails 
that lim„-^oo Rn/n = P^ig -almost surely. Hence, due to the continuity of q 
we have lim„^oo ^^ioQiRn/n) = q{0) = q{0) < lim^j^o fa^i't)/^ = ck- view of 
inequality (5.7), this completes the proof. □ 
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Theorem 5.4. Under the assumptions of Theorem 5.3 let be an 
SUD test of order A„ with liminf„^oo An/?^ > and the condition (5.5) be 
replaced by 

(5.8) Ve > : liminf - F„(p(t))) > [P^] 

for some K G [0, 1] fulfilling K > L = limsup„_,oo A„/n in the case of L <1 
and K =1 otherwise. Supposing that \mixioq{x) exists, we have 

(5.9) Jim FDR,,(v9(„)) = Cnm<7(x) = Cg(0) = ^(0) < C". 

Proof. Again, to avoid triviality, we assume /o(^) 7^ 0- Equation (5.9) 
can be shown by utilizing the notation introduced in the proof of Theo- 
rem 4.3 and the decomposition 

FDR^(y?(„)) = CnY^ q{j/n)P^{Rn = j\pio < aj:n) 
i=i 

n 

+ Cn Yl (l{3/n)Pi){Rn = j\Pio < aj:n) 
J=A„+1 

= M„ + m„, (say). 

In view of Theorem 4.3 and the structure of an SUD test, we obtain by 
applying (4.2) that 

M„ = CnE^^o [Q{Rn/n)l{R^/n<\„/n}], 
mn < CnE^-o [^(-Rn/ra)l{i?„/„>A„/„}]- 

From (5.8) it follows that P^-almost surely Fn{p{Xn/n)) < Xn/n < K and 
consequently Rn/n < Xn/n < K holds true for eventually all n G N. There- 
fore, again due to (5.8), in analogy to the proof of Theorem 5.3 we conclude 
that lim^^oo Rn/n = P^iQ -almost surely, which finally entails 
lim„^ 

oo l{ij„/n>An/n} — P^iQ-aXvoost surely. Together with the bounded- 
ness of q this entails that limyj_>Qo m„ = 0. Moreover, exploiting the continu- 
ity of g at X = we see that lim^^oo Afn = C?(0) = C^(0)) which altogether 
yields the desired result. □ 

Remark 5.1. One cannot expect to obtain exact values for the limiting 
FDR under the quite general assumptions of Theorem 5.2 if r* = 0. To see 
this, consider the case Cn = 1 in which the FDR is equal to the familywise 
error rate (FWER). For = 1 it was shown in [12] that the FWER is equal 
to a for any n G N in the case of a linear SU procedure, while it tends to 
1 — exp(— a) < a for a linear SD procedure. We therefore have to know more 
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about the structure of the underlying procedure in order to compute the 
hmiting FDR in case of r* = 0. The hmiting behavior for procedures based 
on fa (or its modifications) satisfying the assumptions of Theorem 5.4 is 
in accordance with the hnear SU procedure and should be expected, since 
the difference of the critical values ai-n — ia/n tends to zero for i S o(n). 
Therefore, the local behavior around zero should not differ much for large 
n. 

Corollary 5.1 (Examples 3.1-3.3 continued). Assume the distribu- 
tional assumptions (T)3), (11) and (12) hold. Then the SUD procedure based 
on fa with parameter A G [0, 1) and the SU procedures based on fa\, ^ = 1,2, 
as well as the truncated SU procedure asymptotically control the FDR at 
level a. More precisely, if condition (5.3) is fulfilled, that is, lim„__»oo Cn = 
Cg [0,1], then: 

(i) For the SUD procedure the upper bound a for the limiting FDR is 
sharp for ^ G [a, 1]. 

(ii) For the SU procedures based on fa^n, i = 1,2, the upper hound a for 
the limiting FDR is sharp for C, > C*(^) = ^/(^(l — a) + a). In the case of(< 
C*(k), an upper bound for the asymptotic FDR is given by C^c/(-'- ~ C + C^c)' 
where t^ denotes the unique solution of the equation Foo(i|C) = hi(t),i = 1,2, 
on {Ojt^). For finite n, the upper bound given in (4-9) is sharp. 

(iii) For the truncated SU procedure the upper bound a for the limiting 
FDR is sharp for ^ > C*('^)- -^^ the case of Q < C*(^)> upper bound for 
the asymptotic FDR is given by C,k,/{1 — C + C^)- 

Proof. First of all, as mentioned before, a step-up-down test has the 
structural properties (Tl), (T2) and (Al). Moreover, assumptions (D3), (II) 
and (12) imply the crucial monotonicity properties (Dl) and (D2) for a step- 
up-down test. Hence, in order to apply Theorem 5.1, it remains to check the 
validity of condition (5.1). To this end, for notational convenience and with- 
out loss of generality, we work under condition (5.3). We make use of (2.1), 

that is, that the ecdf F„ converges P// -almost surely to its limit -Foo('lC) 
uniformly in t G [0,1]. Since under (Tl) and (T2) we have the identity 
Fn{p{Rn/n)) = Rn/n for all n G N, (2.1) leads to lim„^oo(i^oo(p(^n/n)|C) - 
Rn/n) = Pf/-almost surely. From this we conclude that (P/^-almost surely) 
the only possible accumulation points of the sequence {Rn/n)n£^ consist of 
the solutions of the equation -Foo(p(i)|C) = t in t G [0, 1]. If, as in Examples 3.2 
and 3.3, this solution is unique, then the sequence (i?„/n)„gN necessarily 
converges to this solution r* = r*{(^) G [0, 1] (say) P//- almost surely. 

If, however, as in Example 3.1, the equation Foo{p{t)\C) = t has the so- 
lution t = 1 and exactly one further (smaller) solution in [0, 1), we have to 
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exclude t = 1 cis 3, possible accumula-tion point of (^Rn/'^')nG 

N in the latter case 

in order to prove the P//-almost sure convergence of {Rn/n)n<=N to the small- 
est solution r* = r*{Q (say) of the aforementioned equation. To this end, we 
only consider values of C leading to the two distinct solutions r*(^) and 1. For 
critical value functions p with p{t) < fa^{t) for all t G [0, 1] it is then evident 
that Foo(/o(t)|C) < t for all t € {r* ,1). Moreover, notice that, by definition of 
A„(A), we have the inequalities An(A) — 1 < nr(A) < A„(A) for all n G N. Now, 
if A > p{r*), this, together with condition (2.1), yields that Pf/-almost surely 

-fn(/o(An(A)/7T,)) < \n{X)/n and consequently Rn < A„(A) holds true for even- 
tually all n G N. This entails limsnpj^_^^ Rn/n < limsup„_>(^ A,i(A)/n = 
r(A) < 1 Pj/ -almost surely, which is just what we wanted to show. 

Finally, if A < p{r*), we may choose a A' > p{r*) and compare the number 
of rejections of the corresponding SUD procedures. Since this number is non- 
decreasing in A G [0, 1], we eventually arrive at limsup„^j^ Rn/n < r(A') < 1 
P// -almost surely. 

Since for all procedures under investigation it holds p{t) < fa^{t) for all 
t G [0, 1], we conclude that r* = r*(C) < faitc)- Hence, Theorem 5.1 applies. 
As a consequence, the example procedures asymptotically control the FDR. 

In the case of the SUD procedure in (i), we use p = and obtain 
I"* = fait()- Hence, the upper bound a for the asymptotic FDR is sharp in 
(i) under Dirac-uniform configurations. The sharpness of the upper bound 
a for the asymptotic FDR in (ii) and (iii) is due to the fact that under 
Dirac-uniform configurations with C > C*('^) we obtain r* = fait^)- 

Finally, the sharpness of the upper bounds for the finite n FDR in (ii) 

is a consequence of (A2), which is fulfilled for fa^K, i = 1,2. Sharpness here 
means that the upper bound given in (4.9) is exactly attained for finite n 
under Dirac-uniform configurations. □ 

The latter corollary means, in other words, that procedures based on fa 
fulfilling the assumptions of Theorem 5.2 asymptotically exhaust the whole 
FDR level a under Dirac-uniform configurations. Moreover, the rejection 
curve fa cannot be improved in the sense of the following theorem, which is 
another consequence of Theorem 5.1. In order to formalize this, let a G (0, 1), 
A G [0, 1] and ^A\ denote the set of rejection curves r with the property that 
for all /q ^ with lim„^oo Cn = C for some G [0,1] it holds 

(5.10) limsupFDR,„(^f^°(^)) < limsupsupFDR^c^f^^^^^) < a, 

where ^^x^^^'^^ is the step-up-down procedure of order An = A„,(A) based on 
r. It should be noted that the first inequality in (5.10) is not very restrictive 
since many statistical models satisfy the "model continuity assumptions" 
(SA) formulated in [11], due to which, at least for SUD procedures such 
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as %^x^^^\ the corresponding FDR values FDRjg{ip^^^^^) can be approx- 
imated arbitrarily closely by the values FDR^((/3^^^^^^) for some suitably 
chosen i? € 0, n e N. 

In terms of power it is immediately clear that, whenever ri,r2 S A4x 
with ri < r2, then ^^^^^^^ > ^^x^^^^^ ■ Therefore, a smaller rejection curve 
typically leads to a more powerful test procedure in the sense that more 
(never less) false hypotheses can be rejected. Here we define the power of a 
test (/?(„) by /3tf(i/3(„)) =E^[(i?„ - K)/(ni V 1)]. 

Theorem 5.5 (Asymptotic optimality of /„). 

(i) Lei AG [0,1] and r ^ M\. Then 

(5.11) VtG[0,A]:r(t)>/„(t). 
If X< 1, then it holds for any r € (A, 1] that 

(5.12) Vt G (A,r] :r(t) < /„(t) ^ Vt G (A,r] :r(t) = /„(t). 

(ii) If X <1 and r G A^a smc/i i/iat, for every Q G (a, 1), the equation 
Foo{p{t)\C) = 1 ~ C + Cp(^) = ^ ^^'5 most one solution in (0, 1), it even 
holds r{t) > fa{t) for all t G [0, 1] . 

(iii) If X = l and assuming (D3), (II) and (12), it holds that 

inf r = fa. 

reMi 

Moreover, for any G 0k = {t? G : lim inf „^oo Cn(^?) > a/{K{l — a) +a)}, 
K G (0, 1), the power of any fa € A4i with fait) = fait) for all t G [0, k] is 
asymptotically not smaller than the power of any other r £ A4i, that is, 

(5.13) liminf[^^((^SUD(/.))_^^(^SUD(r))]>Q for all 



Proof. In order to prove part (i), assume that for an arbitrary chosen 
rejection curve r G M.x it holds r(i*) < fait*) for some t* G (0, A). Consider 
now a Dirac-uniform configuration P/p with lim„_»oo Cn = C C ^ (o^i 1) 
chosen such that r(t*) < Foo(t*|C) < fait*)- Then it is obvious that prop- 
erty (5.10) is violated, because (with self-explaining notation) it follows P/q- 
almost surely 

hm^inf pM/n > Foo(t*|C) > F^it^C) = /a(tc) 
and consequently 

hm^inf FDR,„((^^|i°(^)) > Ct*/{1 - C + Ct*) > (tc/il - C + C^c) = 
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due to the fact that the function C,x/{\ — Q + C,x) is strictly increasing 
in X S (0, 1) and t* > tc_. Hence, for all t G (0, A) we have r{t) > fa{i), from 
which the assertion follows. 

Now assume that we have r{t) < fa{t) for all t € (A,r] and r(t*) < 
for some t* S (A,t). Consider now the Dirac-uniform asymptotic model 
DUoo(C*) with C* G (a,l) chosen such that /«(A) < Foo(A|C*), -Foo(ilC*) < 
fa{t*) and ini x<t<t* {Foo{t\C*) ~ > Oi which is possible due to the left- 
continuity of the rejection curve r. Then the argumentation is the same as 
before. Part (ii) and the first assertion of part (iii) can be proven similarly. 

For the proof of (5.13), we assume (in order to avoid triviality) ni(n) > 
for all n € N, define 5„ = i?„ — V„ and denote the set of all /q, G A^i with 
fa{t) = fa{t) for all t € [0,«;] by S^- Then we have (with self-explaining 
notation as before) the inequality 

Vn € N : V/„ e 5. : Vr G All : - ^) l^,, > 0, 

V ni ni J ^ '- ' 

which holds true due to (5.11) and the fact that Sn is nondecreasing in t* . 
Now, for fixed € G^, we utilize the chain of inequalities 

which holds P^-almost surely for eventually all n G N, leading to limsup„^o^ 
(r|P^) < K and consequently to Ij^* (j,)<^} — > 1 [P^] for all i? G Q^- Therefore, 
we obtain P^-almost surely 

(5.14) liminf f^^^i^-^^) >0 for all i? G G«, G r G 

n->oo \ ni ni J 

Taking expectation in (5.14) and utilizing Fatou's lemma, we finally arrive 
at assertion (5.13). □ 

Theorem 5.5 shows that in the class of SU procedures with rejection curve 
r G 1 we always have r > fa- In the class of truncated SU procedures with 
parameter k G (0, 1), the truncated procedure based on is the best choice. 
More generally, if we restrict attention to the subspace G^ C G described 
in (iii) of Theorem 5.5, fa is the asymptotically uniformly best choice on 
[0, k] for a step-up procedure. For SUD procedures with parameter A < 1, 
fa leads to the asymptotically uniformly best choice of critical values on 
the step-up part, see (5.11). On the step-down part of a SUD procedure, 
fa cannot be uniformly improved by some r G A4x whatever r does on the 
step-up part; see (5.12) with r = 1. For arbitrary r G (A, 1], assertion (5.12) 
states that a rejection curve r G Aix cannot be first smaller and then larger 
than fa on the interval (A,l]. It seems possible that A4x contains an r 
which is first larger and then smaller on the step-down part. But this would 
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imply that the SUD procedure based on r is asymptotically less powerful 
than the SUD procedure based on on some Q^- If we restrict attention 
to rejection curves r € M\ described in (ii) of Theorem 5.5, then fa is the 
best choice. These considerations may justify calling the asymptotically 
optimal rejection curve (AORC). 

6. Concluding remarks. In view of the asymptotic optimality results de- 
veloped in Section 5 concerning procedures based on or its modifications, 
it is natural to ask how large n has to become in order to achieve a reason- 
able behavior of the FDR of the proposed procedures. As already mentioned 
in Example 3.1, the asymptotic exhaustion of the whole FDR level has to 
be traded off with a slightly liberal behavior of the procedures based on 
in the finite case. In order to illustrate this effect, we consider the SU proce- 
dures based on i = l,2, where the upper bound given in (4.9) is sharp 
in the DU„(^n)-iiiodel. Due to the pointwise order of these two rejection 

(2) 

curves (see Figure 1) it is clear that an SU procedure based on fa,K is more 
liberal in the DUn(Cn)-iiiodel. We therefore present results for this proce- 
dure. Figure 2 depicts the behavior of this procedure under Dirac-uniform 
configurations with a varying number of true hypotheses for three differ- 
ent values of n. For n = 100, there is a notable violation of the FDR level 
a = 5% for 12 < no < 35. The largest FDR under Dirac-uniform is attained 
for no = 16 with numerical value 0.05801. For the two larger values of n, 
the actual level does not exceed a by much. Computation of the FDR for a 
SU(D) procedure in the case of an underlying Dirac-uniform configuration 
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Fig. 2. Actual FDR of the SU procedure based on /q^Jj forn = 100, 500, 1000 depending 
on no under Dirac-uniform configurations. The curves can be distinguished by noticing that 
the maximum FDR becomes smaller for increasing n. 



can be done by utilizing formulas for the joint distribution function of order 
statistics, see [21], pages 366-367, and [10]. 

We will give one brief suggestion for a modification of fa in the finite 
case. However, this shall not be emphasized too much, because on the one 
hand, the AORC is designed for the asymptotic case and on the other hand, 
we have to keep in mind that the FDR values under Dirac-uniform reflect 
an unrealistic worst case scenario. For realistic alternatives, we get much 
smaller realized FDRs so that the original AORC may safely be used in the 
finite case, for example, n > 500. 

We only mention one possibility to obtain a valid set of critical values for 
an SU or SUD procedure guaranteeing strict FDR control, that is, we adjust 
the critical values given in (3.1) in an appropriate way. For example, we can 
try to find a suitable /3„ > such that the choice 

(6.1) ai:n = — —5 7- r, i = l,...,n, 

yields an SU procedure (or SUD procedure) controlling the FDR at level a. 
The critical values (6.1) correspond to the rejection curve 

/a(t)= (l + ^)/a(t), tG[0,a/ia + f3n/n)]. 

It is remarkable that herewith a direct connection to the considerations in 
[3] can be drawn. In the remark to Definition 7 in [3], the authors propose an 
(adaptive) SD procedure with critical values given by (6.1) and the universal 
adjustment constant /?„ = 1.0. FDR control for this SD procedure is proved 
in [15] in case of independent values. For an SU procedure, the adjustment 
constant has to be larger. For example, for a = 0.05, an SU procedure with 
n = 100 and the choice /3ioo = 1-76 leads to strict FDR control. 
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A systematic comparison of other procedures controlling the FDR strictly 
or asymptotically with procedures based on the AORC goes beyond the 
scope of this paper and is directed to future research. Typically, each ap- 
proach has assets and drawbacks. At least in the class of SUD procedures 
(including SD and SU procedures) based on fixed rejection curves the new 
procedures based on the AORC are powerful alternatives. One can construct 
situations, especially if the proportion ( of true hypotheses is small, where 
the new procedures reject many more hypotheses than the LSU procedure. 
On the other hand, if C is large, there may be only a few more or, in rare 
cases, fewer rejections. In other words, if C is close to one, it is hard to beat 
the LSU procedure by a considerable amount. Moreover, it should not be 
concealed that the LSU procedure has the advantage that it applies for cer- 
tain situations with positive dependent p-values, whereas procedures based 
on the AORC may fail to control the FDR in such situations. For exam- 
ple, this is the case for p- values based on normally distributed test statistics 
with positive correlation as it appears in multiple comparisons with a con- 
trol. Whether the AORC works for pairwise comparisons or multiple testing 
of correlation coefficients pij {Hij : pij = versus Kij : pij ^0, I < i < j < k) 
is currently under investigation. 

Finally, one may think about a more flexible concept of FDR control 
depending on the proportion = Cn{'&) of true hypotheses which allows, 
for example, a larger FDR for larger values of and a smaller FDR for 
smaller values of C„, or vice versa. Therefore, one may choose a suitable 
FDR controlling function g: [0,1] — > [0, 1] and require FDR control at this 
level function g, that is, 

Vi?Ge:FDR^(v9(„))<5(C„W). 

This means that for any ^„ G [0,1] the FDR is controlled at level g{Cn) if 
the proportion of true hypotheses is Cn • Obviously, for the LSU procedure at 
FDR level a we can choose g{C) = Cct- For an SUD procedure related to the 
AORC with critical values defined in (6.1) and suitable (3n we can choose 
g{C) =min{Q,C}. Asymptotic FDR control at level function g now means 
that 

limsupsup(FDR^(((C)(„)) - g{Cn{^))) < 0. 

For example, in the case of the truncated SU procedure, we get from Corol- 
lary 5.1 that we can choose g{C) = q for C G [C*{'^)i 1] and g{C) = C'^/(l — 
C + C^) for Cg[0,C('^)). 
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